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(57) Abstract: A system :ind method lor recording and reconstructing a high definition video on a standard definition DVD. A 
recording system is provided that comprises: a system for scaling down an I ID video to an SD video format; a system for encoding 
the SD video; a system for extracting enhancement information from the HD video; and a system for storing the SD video and 
the extracted enhancement information onto the SD compatible medium. A playback system is provided for reconstructing the I ID 
video image from the standard definition SD format recording, comprising: a system for extracting and decoding SD data from the 
recording; a system for extracting enhancement information from the recording; a system for de-interlacing the decoded SD data; 
and a system for up-scaling and post -processing the decoded SD data with the enhancement information to generate the I ID video 
image. 
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1 

System and method for providing high definition material on a standard definition compatible 
medium 



V 



The present invention relates generally to systems for recording and playing 
back digital image data, and more particularly relates to a system and method for recording 
high definition material on a standard definition compatible DVD so that the DVD can be 
played back in either a standard or high definition mode. 



As DVD (digital versatile disk) players become the medium of choice for 
video playback, and eventually video recording, the demand for DVD systems having 
advanced features is expected to grow. For instance, various companies, including Philips 

10 Electronics, have recently launched DVD recorders into the marketplace. As DVD recorders 
become more popular in the future, recording TV programs and the like on DVDs will 
become easier and more affordable. 

One of the issues, however, that has yet to be adequately addressed relates to 
the increasing popularity of high definition (HD) material. Presently, there is no set format 

15 for using standard definition (SD), i.e., 4.7 GB, single sided, DVDs for recording and playing 
back high definition (HD) material (referred to herein as HD-on-SD-DVD). This issue will 
only become more important as 2006 approaches, which is when the Advanced Television 
Systems Committee (ATSC) has mandated that high definition broadcasts become the 
standard for U.S. television channels. 

20 Based on the above, it can be seen that recording HD video in a standard, i.e., 

SD-DVD format, will become an important option for DVD recorders and open new markets 
for consumer electronics, semiconductors and film industries. Furthermore, techniques for 
providing HD-on-SD-DVD will become useful for other applications and mediums that 
support long play mode recording, such as HD-DVD recorders or hard-disk based recorders. 

25 Accordingly, a need exists for a system that can provide HD material on a SD 

compatible medium, such as an SD-DVD. 
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The present invention addresses the above-mentioned issues, as well as others, 
by providing a system and method for providing HD recording and playback systems that 
provide an HD feel on an up-converted SD image using enhancement information extracted 
from the original HD image during recording. 
5 In a first aspect, the invention provides a recording system for recording a high 

definition video onto a standard definition compatible medium, comprising: a system for 
scaling down the HD video to an SD video format; a system for encoding the SD video; a 
system for extracting enhancement information from the HD video; and a system for storing 
the SD video and the extracted enhancement information onto the SD compatible medium. 

10 In a second aspect, the invention provides a playback system for 

reconstructing a high definition video image from a standard definition format recording, 
comprising: a system for extracting and decoding SD data from the recording; a system for 
extracting enhancement information from the recording; a system for de-interlacing the 
decoded SD data; and a system for up-scaling and post-processing the decoded SD data with 

1 5 the enhancement information to generate the HD video image. 

In a third aspect, the invention provides a method for recording high definition 
(HD) video onto a standard definition compatible medium, comprising: scaling down the HD 
video to an SD video format; encoding the SD video; generating enhancement information 
from the HD video; and storing the SD video and the enhancement information onto the SD 

20 compatible medium. 

In a fourth aspect, the invention provides a method of reconstructing a high 
definition video image from a standard definition format recording, comprising: extracting 
and decoding SD data from the recording; extracting enhancement information from the 
recording; de-interlacing the decoded SD data; and up-scaling and post-processing the 

25 decoded SD data with the enhancement information to generate the HD video image. 

In a fifth aspect, the invention provides a program product stored on a 
recordable medium for recording high definition video onto a standard definition DVD, 
comprising: means for scaling down the HD video to an SD format video; means for 
encoding the SD video; means for generating enhancement information from the HD video, 

30 wherein the enhancement data comprises high frequency image data; and means for storing 
the SD format video and the enhancement information onto the DVD. 

In a sixth aspect, the invention provides a program product stored on a 
recordable medium for reconstructing a high definition video image from a standard 
definition DVD, comprising: means for extracting and decoding SD data from the DVD; 
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means for extracting enhancement information from the DVD, wherein the enhancement 
information is stored in an MPEG userdata field and comprises high frequency image data; 
means for de-interlacing the decoded SD data; and means for up-scaling and post-processing 
the decoded SD data with the enhancement information to generate the HD video image. 



These and other features of this invention will be more readily understood 
from the following detailed description of the various aspects of the invention taken in 
conjunction with the accompanying drawings in which: 
10 Figure 1 depicts a recording system in accordance with the present invention. 

Figure 2 depicts a playback system in accordance with the present invention. 



The present invention provides a video processing system and method that can 
15 record HD material in an SD compatible DVD (SD-DVD) in a manner such that the DVD 
can be played back in a regular DVD player for regular SD viewing, or in an HD enabled 
DVD player for HD viewing. Any type of writable or re-writable DVD (e.g., DVD-R, -RW, 
+RW, -RAM) may be utilized. Using the techniques described herein, the system is able to 
record a two-hour "near" HD quality video in a 4.7 GByte DVD, or a four-hour near HD 
2 C - video in a 9 Gbyte DVD. As will be explained in further detail below, during recording, the 
system both: (1) converts the inputted HD signal into an SD signal; and (2) generates 
enhancement information from the HD signal. The system then encodes the SD signal, e.g., 
using an MPEG-2 encoder, and stores the enhancement information, e.g., in the userdata field 
of the MPEG bitstream. 

25 The exemplary embodiments described herein utilize only a relatively small 

amount of enhancement information that captures important HD image features, such as edge 
and texture infomiation. hi this manner, a low bitrate can be used for the enhancement 
information (i.e., less than 0.5 Megabytes/second) and the combination of the SD signal and 
enhancement information can be maintained at approximately 5 Megabytes/second. Prior art 

30 approaches, such as MPEG-2 layered or scalable coding, could not maintain such a low 
bitrate at a reasonable quality. 

During playback, the recorded DVD is not only fully compatible to current 
DVD standards, but the DVD can also be played back in an HD enabled DVD player that 
uses the enhancement data to generate an HD quality video. Note that while the 
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embodiments provided herein in Figure 1 and 2 describe the SD format output 24 as a DVD, 
it should be understood that any recordable medium could be utilized. 

Referring now to Figure 1, an exemplary recording system 10 in accordance 
with the present invention is shown. System 10 receives an HD input signal 1 1 and generates 
5 an SD format output 24. The input of the system may accept all ATSC formats either in 

bitstream form or in component-signal form (for the bitstream form, an HD MPEG-2 decoder 
would be required). However, for the purpose of explanation, system 10 is shown accepting 
1080i (interlaced) and 720p (progressive) formats. Accordingly, the exemplary system is 
capable of processing two different formats, progressive and interlaced, both of which are 
10 compatible with current SD standards. The progressive format is noted as 30p or 60i 

prog(ressive) (similar to film mode), and they are in the upper part of flow arrows after the 
"prog-to-interlace-converter" block 16. Note that 60i prog is structurally the same as 30p, 
except that 60i prog is treated as the interlaced format. The reason for converting 3 Op to the 
60i progressive format is to guarantee the compatibility of the recorded DVD for all the DVD 
15 players that support (re)wri table DVDs. The interlaced format is noted as 60i, and it is in the 
lower part of the flow arrows. 

The basic operation of the recording system 10 is as follows. The input 
pictures (i.e., video) are scaled down to SD pictures, and some important HD features or HD 
enhancement data (referred to herein as E-data) is extracted from the input. An MPEG-2 
20 encoder then codes the SD sequences, and the MPEG-2 bitstream is saved in storage (e.g., 

DVD+RW) together with E-data. The E-data can be stored in the userdata field of the MPEG 
bitstream. 

System 10 comprises a de-interlacer / rate subsampling system 12 that 
deinterlaces or subsamples the input signal 1 1 to 30p or 60p formats. Down-conversion/ 
25 aspect ratio (AR) formatting system 14 then formats the signal with a widescreen, 

letterboxing, or expansion (pan and scan) aspect ratio format. Progressive-to-interlace 
converter 16 performs a 2-2 pull-down from 30p to 60i prog, or performs interlacing to 
convert 60p to 60i. MPEG encoder 18 then encodes the signal into an NTSC or PAL 
compatible format. 

30 In addition, after the input signal is de-interlaced/subsampled, the signal is also 

passed to an HD detail extraction system 20, which extracts high frequency image data from 
the signal. The extraction may be accomplished with, for instance, a high pass filter or 
residual operator. The high frequency image data is then passed to an HD feature processing 
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system 22 to generate HD enhancement information, or E-data. The E-data is then stored 
with the SD format output 24, for instance, in the userdata field of the MPEG encoded data. 

For the purposes of this invention, it should be understood that no limitations 
exist on the type of E-data that may be generated and used. It is recognized however that 
5 high frequency image (HFI) data, which provides important edge and textual details, are 
important in generating an HD image. Unfortunately, encoding those regions bit by bit will 
significantly increase the total bitrate, which must be kept to around 5 Mbps if, e.g., a two 
hour video is to be stored on 4.7 Gbyte DVD. As an alternative, the present exemplary 
embodiment proposes to create an HD "feel" by using a multilevel sharpness enhancement 

10 on those regions. 

To achieve this, a multigrid coding system within feature processing system 
22 may be applied to identify and code energy significant regions using quadtree 
decomposition. A resulting region classification bitstream (i.e., an energy region map) will 
then form the E-data, which can be used to generate a gradual gain map by the decoder / 

15 playback system. In accordance with the coding system, more important areas of an image 
are identified and are broken into smaller and smaller blocks or regions. The partition 
process follows the quadtree structure and produces a string of 1 's and 0's. A multilevel 
region classification algorithm for generating an energy region map may be implemented as 
follows: 

20 Partition the HFI into 16x16 blocks. Then calculate the total energy (E, for the 

textual test) in each block and the difference (D, for the edge tests) between the average of 
the first three maximum pixel values and the average of the first three minimum pixel values 
in each block. 

Apply the quadtree decomposition with 128x128 blocks in the HFI first. If E 
25 > threshold, or D > threshold 2 in any 16x16 blocks within a 128x128 block, the 128x128 
block is marked as bit "1" and is then divided into 4 64x64 blocks; otherwise, the block is 
marked "0" without any further partition. The decomposition continues until 16x16 blocks 
are reached. 

Test all 16x16 blocks with the thresholds and mark blocks "1" if E or D is 
30 greater than the thresholds plus a constant to further classify 16x16 blocks; otherwise, mark 
them "0". Store all the "0"s and "l"s of the decomposition and this 16x16 block test as E- 
data. Depending on how dense the energy spreads are in the HFI, there may exist different 
bit sizes in E-data. For example, a typical 720x1 2S0 HFI can be classified into 5 levels using 
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less than 1000 bits. This is around 30Kbps or 0.03 Mbps, which is very little compared with 
5 Mbps of the SD layer. 

It should be understood that the above algorithm is just one example of a 
system for generating E-data, and other methods, now known or later developed are within 
5 the scope of this invention. 

During playback, the bitstream in the DVD can be decoded by any DVD 
compliant decoder in an SD format. In addition, for an HD-enabled decoder, the E-data 
(stored, e.g., as userdata) is retrieved, post-processed, and combined with the regular SD 
upconverted pictures to create an image that can be displayed on an HD-ready TV with an 
10 HD "feel." 

hi accordance with the quadtree algorithm method described above, during 
post-processing, the quadtree is reconstructed and a gain is assigned to each of the levels to 
create a gradual gain map. Thus, for example, different gains are assigned to the different 
block sizes or partition regions according to the quadtree structure. With the different level 
15 of enhancement gain, an HD "feel" will be created if a sharpness enhancement algorithm was 
applied. 

Specifically, the partition regions are reconstructed by decoding the quadtree 
bitstream in E-data. Each region is then assigned a value based largely on the size of the 
region. Thus, for example, 128x128 block regions are assigned 0.5; 64x64 block regions are 
20 assigned 0.7; 32x32 block regions are assigned 0.8; and 16x16 block regions are assigned 
either 0.9 or 1 .0. The result provides a gain map that can be used by a playback system 
(Figure 2) to restore HD effects on an upconverted SD picture. 

Any type of enhancement method may be utilized in conjunction with the gain 
map reconstructed from the E-data. Exemplary methods include adaptive peaking and/or LTI 
25 (Luminance Transient Improvement). 

Referring now to Figure 2, a playback system 30 is shown for playing back 
image data stored on DVD 24. Playback system 30 comprises an MPEG decoder 32 for 
generating either a 60i progressive or 60i signal and a de-interlacer 34 for generating either a 
30p or 60p signal. An upconversion and postprocessing system 36 receives the 30p or 60p 
30 signals, as well as the E-data, to generate HD 30p or HD 60p signals. A frame rate doubler 
or interlacer system 38 then generates either a 1080i or 720@ 60p output. 

It is understood that the systems, functions, mechanisms, methods, and 
modules described herein can be implemented in hardware, software, or a combination of 
hardware and software. They may be implemented by any type of computer system or other 
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apparatus adapted for carrying out the methods described herein. A typical combination of 
hardware and software could be a general-purpose computer system with a computer 
program that, when loaded and executed, controls the computer system such that it carries out 
the methods described herein. Alternatively, a specific use computer, containing specialized 
5 hardware for carrying out one or more of the functional tasks of the invention could be 

utilized. The present invention can also be embedded in a computer program product, which 
comprises all the features enabling the implementation of the methods and functions 
described herein, and which - when loaded in a computer system - is able to carry out these 
methods and functions. Computer program, software program, program, program product, or 
10 software, in the present context mean any expression, in any language, code or notation, of a 
set of instructions intended to cause a system having an information processing capability to 
perform a particular function either directly or after either or both of the following: (a) 
conversion to another language, code or notation; and/or (b) reproduction in a different 
material fomi. 

15 The foregoing description of the preferred embodiments of the invention has 

been presented for purposes of illustration and description. They are not intended to be 
exhaustive or to limit the invention to the precise form disclosed, and obviously many 
modifications and variations are possible in light of the above teachings. Such modifications 
and variations that are apparent to a person skilled in the art are intended to be included 

20 within the scope of this invention as defined by the accompanying claims. 
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CLAIMS: 



1 . A recording system [10] for recording a high definition (HD) video onto a 
standard definition (SD) compatible medium [24], comprising: 

a system [14] for scaling down the HD video to an SD video format; 
a system for encoding [18] the SD video; 
5 a system for extracting enhancement information [20] from the HD video; and 

a system for storing the SD video and the extracted enhancement information 
onto the SD compatible medium [24]. 

2. The recording system [10] of claim 1, wherein the SD compatible medium 
1 0 [24] comprises a DVD. 

3. The recording system [10] of claim 1, wherein the system for encoding [18] 
comprises an MPEG-2 encoder. 

15 4. The recording system [10] of claim 1, wherein the enhancement information 

comprises high frequency image data partitioned into different energy regions. 

5 . The recording system [ 1 0] of claim 4, wherein enhancement information is 
stored in an energy region map by a feature processing system [22]. 

20 

6. The recording system [10] of claim 5, wherein the energy region map is coded 
using a quadtree decomposition algorithm. 

7. The recording system [10] of claim 1, wherein the SD video and the extracted 
25 enhancement information are stored at a combined rate of approximately 5 megabits/second. 

8. The recording system [10] of claim 1, wherein the extracted enhancement 
information is stored at a rate of less than 1 megabit/second. 
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9. The recording system [10] of claim 1, further comprising an aspect ratio 

format system for formatting the SD video for widescreen, letterboxing, and scan and pan 
formats. 



10. The recording system [10] of claim 1, wherein the SD video can be stored in a 

format selected from the group consisting of: progressive and interlaced. 



11. The recording system [10] of claim 1, wherein the enhancement information is 

stored in an MPEG userdata field. 



12. A playback system [30] for reconstructing a high definition (HD) video image 

from a standard definition (SD) format recording [24], comprising: 

a system [32] for extracting and decoding SD data from the recording [24]; 

a system for extracting enhancement information from the recording [24] ; 
15 a system [34] for de-interlacing the decoded SD data; and 

a system for up-scaling and post-processing [36] the decoded SD data with the 
enhancement information to generate the HD video image. 



13. The playback system [30] of claim 12, wherein the enhancement information 

20 comprises information relating to high frequency image data extracted during a recording 
process. 



14. The playback system [30] of claim 13, wherein the enhancement information 
comprises an energy region map. 

25 

15. The playback system [30] of claim 14, wherein the energy region map is 
coded with a quadtree decomposition algorithm. 

16. The playback system [30] of claim 14, wherein the post-processing system 
30 [36] applies adaptive peaking with a gain map derived from the enhancement information. 



17. The playback system [30] of claim 14, wherein the post-processing system 

[36] applies luminance transient improvement with a gain map derived from the enhancement 
information. 
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18. A method for recording high definition (HD) video onto a standard definition 

(SD) compatible medium [24], comprising: 

scaling down the HD video to an SD video format; 
5 encoding the SD video; 

generating enhancement information from the HD video; and 

storing the SD video and the enhancement information onto the SD compatible 

medium [24]. 

10 19. The method of claim 1 8, wherein the step of generating enhancement 

information includes: 

extracting high frequency image data from the HD video; and 
creating a energy region map based on the high frequency image data using a 
quadtree algorithm. 

15 

20. The method of claim 1 8, wherein the enhancement information is stored in an 
MPEG userdata field. 

21. A method of reconstructing a high definition (HD) video image from a 
20 standard definition (SD) format recording, comprising: 

extracting and decoding SD data from the recording; 
extracting enhancement information from the recording; 
de-interlacing the decoded SD data; and 

up-scaling and post-processing the decoded SD data with the enhancement 
25 information to generate the HD video image. 

22. The method of claim 21, wherein the enhancement information is stored in an 
MPEG userdata field and comprises an energy region map coded with a quadtree 
decomposition algorithm. 

30 

23. The method of claim 22, wherein the decoded SD data is post-processed by 
applying adaptive peaking with a gain map derived from the enhancement infomiation. 
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24. The method of claim 22, wherein the decoded SD data is post-processed by 

applying luminance transient improvement with a gain map derived from the enhancement 
information. 

5 25. A program product stored on a recordable medium for recording high 

definition (HD) video [11] onto a standard definition (SD) DVD [24], comprising: 

means [14] for scaling down the HD video to an SD format video; 
means [18] for encoding the SD video; 

means [20, 22] for generating enhancement information from the HD video, 
1 0 wherein the enhancement data comprises high frequency image data; and 

means for storing the SD foimat video and the enhancement information onto 

the DVD. 



26. A program product stored on a recordable medium for reconstructing a high 
15 definition (HD) video image from a standard definition (SD) DVD [24], comprising: 

means [32] for extracting and decoding SD data from the DVD; 

means for extracting enhancement information from the DVD, wherein the 
enhancement information is stored in an MPEG userdata field and comprises high frequency 
image data; 

20 means [34] for de-interlacing the decoded SD data; and 

means [36] for up-scaling and post-processing the decoded SD data with the 
enhancement infomiation to generate the HD video image. 

27. The program product of claim 26, wherein the post-processing means [36] 
25 applies one of the group consisting of: adaptive peaking and luminance transient 

improvement, with a gain map derived from the enhancement information. 
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